我们研究保形预测的鲁棒性,这是标记噪声的不确定性定量的强大工具。我们的分析解决了回归和分类问题,表征了何时以及如何构建正确覆盖未观察到的无噪音地面真相标签的不确定性集。通过风格化的理论示例和实际实验,我们认为天真的保形预测涵盖了无噪声的地面真相标签,除非噪声分布是对手设计的。这使我们相信,除了病理数据分布或噪声源外,对标签噪声的纠正是不必要的。在这种情况下,我们还可以在保形预测算法中校正有界大小的噪声,以确保在没有得分或数据规律性的情况下正确覆盖地面真相标签。
translated by 谷歌翻译
Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty and estimate reliable probabilities. In particular, they tend to be overconfident. We begin to address this problem in the context of multi-class classification by developing a novel training algorithm producing models with more dependable uncertainty estimates, without sacrificing predictive power. The idea is to mitigate overconfidence by minimizing a loss function, inspired by advances in conformal inference, that quantifies model uncertainty by carefully leveraging hold-out data. Experiments with synthetic and real data demonstrate this method can lead to smaller conformal prediction sets with higher conditional coverage, after exact calibration with hold-out data, compared to state-of-the-art alternatives.
translated by 谷歌翻译